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Development of the Space Operations Incident 
Reporting Tool 


Abstract 

The space operations incident reporting tool (SOIRT) is an instrument used to 
record information about an anomaly occurring during flight which may have been 
due to insufficient and/or inappropriate application of human factors knowledge. 
The SOIRT form was originally developed after researching other incident reporting 
systems of this type. We modified the form after performing several in-house 
reviews and a pilot test to access usability. Finally, crew members from Space 
Shuttle flights STS (space transportation system)-70 and STS-73 participated in a 
usability test of the tool after their missions. 

Since the National Aeronautics and Space Administration (NASA) currently has no 
system for continuous collection of this type of information, the SOIRT was 
developed to report issues such as reach envelope constraints, control operation 
difficulties, and vision impairments. However, if the SOIRT were to become a 
formal NASA process, information from crew members could be collected in a 
database and made available to individuals responsible for improving in-flight 
safety and productivity. Potential benefits include documentation to justify the 
redesign or development of new equipment/systems, provide the mission planners 
with a method for identifying past incidents, justify the development of timelines 
and mission scenarios, and require the creation of more appropriate work/ rest 
cycles. 


1.0 Introduction 

With NASA's increased emphasis on safety and accident prevention, it is important 
to try to identify situations that could result in safety problems in the future. 
Spacesuits that are difficult to don, controls that are difficult to operate, and displays 
that are non-intuitive, while not safety issues themselves, all have the potential of 
impacting mission safety and productivity. 

NASA currently has no system for collecting reports of anomalies specifically 
related to human factors. Therefore, an incident reporting tool, the SOIRT, was 
developed. The SOIRT is a means for describing a human factors incident during 
space operations: what happened, where it happened, who was involved, why it 
happened, and any possible solutions. The SOIRT is intended to be used whenever 
human factors issues are suspect (e.g., reach envelope constrained by hardware, 
controls or switches that are difficult to operate, impaired vision due to glare). The 
SOIRT is also intended to be used to describe instances where human-machine 
interactions can be improved. It is not intended to be used to document incidents or 


1 



events involving bodily harm or injury. This form should be used to compliment 
safety and/or accident reports when appropriate. 


2.0 Purpose 

The SOIRT's purpose is to provide a process for identifying human factors issues 
that impact space operations. Once documented, the incident reports could be 
integrated into a "lessons learned" database or similar-type database. The human 
factors issues identified could then be distributed to the appropriate organizations 
for resolution. 


3.0 Method of Development 

The SOIRT form was developed after researching other incident reporting systems 
in use. A model for the SOIRT exists in the form of the aviation safety reporting 
system (ASRS) program. The ASRS program was designed to gather data regarding 
aviation incidents, and to supplement information derived from aircraft accident 
investigations. During the SOIRT development process, we gathered a number of 
incidents from previous manned space flight missions to serve as test cases for the 
SOIRT. We then compiled the data from a variety of sources such as mission 
reports, flight crew debriefings, and crew questionnaires. 

After several in-house reviews and a preliminary review by a crew representative, 
we performed a non-crew member pilot test to determine the SOIRT's usability. We 
then modified the form based on the non-crew member pilot test results. Finally, 
STS-70 and STS-73 crew members participated in a usability test of the tool after 
their missions. The crew member-suggested modifications to the SOIRT are now in 
progress. 


4.0 The SOIRT Form 

The SOIRT is a tool structured to allow operations personnel, flight controllers, and 
crew members a process for identifying human factors issues that impact space 
operations. (See Appendix A for a copy of the SOIRT used by STS-70 and STS-73 in 
their usability testing). It is divided into three sections after a brief introduction to 
the purpose of the SOIRT. The first section is titled "General Information" and 
contains information regarding personnel, location, time, date, and other 
information pertinent to the incident. The second section of the SOIRT is the 
"Description of Incident" and is a checklist regarding the incident. There are three 
category headings: "Environment," "Human," and "Equipment/Systems." 
Personnel may check any items under each category that apply to the incident (many 
items will overlap in the different categories). An area is provided directly below 
this section to write a description of the incident. The third section of the SOIRT is 
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"Causes & Possible Solutions." There are three categories included in this section: 
minor modifications, significant modifications, and a complete hardware and/or 
software redesign. Personnel should check the appropriate category to define how 
severe the modifications are to preclude reccurrence of the incident. There is also a 
space provided for personnel to identify the cause of the incident, and provide 
suggestions for preventing recurrence of a similar incident in the future. 

Two computerized versions of the SOIRT were developed after crew members 
requested an electronic format. The electronic formats of the SOIRT are in 
SuperCard® for the Macintosh and in Microsoft Visual Basic® for the IBM- 
compatible computer systems. Input fields were used for general information 
regarding personnel, the incident, and roles in the incident. Check boxes were used 
to check items in the description of the incident. On-line definitions were provided 
for the items found in the check boxes by depressing the shift key and clicking on the 
item to be defined. There was also an area provided to type in the description of the 
incident, the cause, and any possible solutions. 


5.0 Testing the SOIRT 

The SOIRT has been tested in three parts: the non-crew member pilot test, the 
STS-70 crew member usability test, and the STS-73 crew member usability test. 


5.1 The Non-Crew Member Pilot Test 

After several in-house reviews and a preliminary review by a crew representative, a 
non-crew member pilot test was performed to determine the SOIRT's usability. The 
objectives of the non-crew member pilot test group were 1) to determine whether 
the main category headings and the items found under each main category heading 
would be useful to a person unfamiliar with human factors in organizing their 
incident description, and 2) to determine and resolve any confusion that might 
result while using the SOIRT due to the directions or procedures of the form. 


5.1.1 Method 

5.1.1.1 Subjects. The non-crew member pilot test group consisted of 13 subjects: 
four co-operative students, two secretaries, two data entry technicians, two electrical 
engineers, one data processor, one human factors specialist, and one human factors 
associate. 

5.1.1.2 Procedure. The subjects were briefed about the purpose of the experiment. 
They were then asked to read six examples of incidents that occurred on previous 
missions. After reading each example, the subjects were asked to identify human 
factors items under the main categories that contributed to the incident. 
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The subjects were also asked to identify and categorize each incident using 
Categories I through III as to whether the incident needed minor modifications, 
significant modifications, or a complete hardware and/or software redesign. 

The subjects answered a short questionnaire after categorizing the six incident 
examples. The questionnaire contained questions regarding the difficulty that each 
subject experienced while classifying the incidents. 

5.1.1.3 Results. We graphed the data from the non-crew member pilot test, each 
example according to the frequency of responses for each category (Appendix B). 
Human factors professionals then evaluated the results independently to fine-tune 
categories, identify inconsistencies across the subject choices, and determine other 
problem areas. Comments from the subjects were also taken into consideration in 
evaluating the SOIRT form. The following table summarizes changes made to the 
form after the usability test. 


Table 1. Revisions to the SOIRT Form After Usability Testing 


ORIGINAL 

REVISED 

REASON 

machine category 

equipment/ systems category 

clarification 

operational category 

deleted 

concepts regrouped 

acoustics /noise 

noise level 

misunderstood 

confined space 

available space 

misunderstood 

atmosphere 

air quality/pressure 

misunderstood 

— 

comfort 

subject suggestion 

cultural differences 

cultural /language differences 

clarification 

language 

deleted 

combined / clarity 

processing/ calculation 

mental processing 

misunderstood 


p rocedures / training 

concepts regrouped 


5.1.1.4 Revisions to Instructions. We made two primary revisions to the SOIRT 
instructions: 1) the importance of the category headings was stressed, and 2) the 
proper use of the categories and items was clarified. In addition, we developed 
definitions and examples for each of the items (to be used as needed). 
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5.2 STS-70 Crew Member Usability Test 


The objectives of the STS-70 crew member test group were 1) to gain usability data 
on the SOIRT using a real environment /incident on the shuttle, 2) to acquire crew 
member input on the feasibility /merit of the SOIRT and 3) to acquire crew member 
ideas on developing the SOIRT into a formal NASA form/process. 

5.2.1 Method 

5.2.1.1 Subjects. The crew member usability test of the SOIRT consisted of the five 
crew members from the STS-70 mission. 

5.2.1.2 Procedure. The SOIRT was given to the crew members of STS-70 after 
returning from their mission. Each crew member was given the choice of testing 
the SOIRT using the paper version or the computerized version. The crew 
members were also asked to answer a questionnaire regarding the SOIRT format 
and process. 

5.2.1.3 Results. Data collected from the questionnaire included crew member 
responses to seven free-format questions (if they used the computerized version of 
the SOIRT, an additional set of six free-format questions was included). The crew 
members concluded that the SOIRT was a much-needed process at NASA. One 
crew member reported that the SOIRT would help to improve human factors in the 
future. When asked if they had any thoughts on how the SOIRT should be 
developed into a formal NASA form/process, one crew member responded that it 
should be integrated into standard debriefs. Another crew member suggested 
modelling the program after the safety form for aircraft incidents. Overall, the crew 
members' responses to the SOIRT were positive and helpful in reinforcing the 
importance of the SOIRT and its need in the space program. 


5.3 STS-73 Crew Member Usability Test 

5.3.1 Method 

5.3.1.1 Subjects. Four out of the seven STS-73 crew members participated in the 
usability test of the SOIRT. 

5.3.1.2 Procedure. The SOIRT was given to the crew members of STS-73 during the 
post-flight debrief. Each crew member was given the choice of testing the SOIRT 
using the paper version or the computerized version. The crew members were also 
asked to answer the same questionnaire given to STS-70 crew members regarding 
the SOIRT format and process. 

5.3.1.3 Results. The crew members considered the SOIRT a very much needed 
project. They recommended changing the format of the SOIRT from an incident 
report to that of a suggestion form. The word "incident" implied that an actual 
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event occurred which would need to be reported. Therefore, any confusion could be 
avoided by replacing the word "incident" with "suggestion." Crew members also 
commented that categorizing the incident in Part II, "Description of Incident," 
helped to structure their report. Other suggestions included broadening the 
categories and adding "reach" and "accessibility" classifications. 

Crew members concluded that the SOIRT should be kept informal and could 
become a part of the crew equipment debrief after each mission. Overall, the crew 
members' responses were very positive, recognizing the need for the SOIRT in the 
space program. 


6.0 Conclusions 

Overall, crew members viewed the SOIRT as acceptable and ready for use. However, 
the crew members' comments and suggestions will be evaluated and incorporated 
before implementing the SOIRT. 

Once implementation begins and information on the SOIRT form is collected from 
crew members, it could be entered into a database. The SOIRT database could 
contribute to more efficient and safe human exploration in microgravity 
environments. Benefits could include not only a process for justifying redesign or 
development of new equipment/ systems, but also for providing mission planners 
with a method for identifying past incidents, developing timelines and mission 
scenarios, and developing schedules for more efficiency and safety. 

The design of the SOIRT database could be applied directly to earth-based activities 
where complex operations and procedures are used. Earth operations that could 
benefit from the incident reporting tool include control rooms (power, chemical 
plant, and other), medical procedures, and submarine operations. 
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Appendix A 



The purpose of the Space Operations Incident Reporting Tool (SOIRT) is to provide a process for 
identifying human factors issues that impact space operations. 


Human factors is defined as discovering and applying information about human capabilities 
and limitations to help in the development of safe, comfortable and effective tools, machines, 
systems, tasks, jobs, and environments. The SOIRT should be used whenever human factors 
issues are suspect (e.g., reach envelope constrained by hardware, controls or switches that are 
difficult to operate, impaired vision due to glare). 


The SOIRT is a means for describing a human factors incident: what happened, where it 
happened, who was involved, why it happened, and any possible solutions. An incident is 
denned as any event or anomaly that occurs during space related operations. Information 
gained from this form will be used to evaluate and improve processes and workplace designs. 

THIS FORM SHOULD BE USED TO DESCRIBE INSTANCES WHERE HUMAN-MACHINE 
INTERACTIONS CAN BE IMPROVED. It should not be used to document incidents or events 
involving bodily harm or injury (use NASA Mishap Form 1627). This form should be used to 
compliment safety and/ or accident reports when appropriate. 


I. General Information 


I 

Name 

Job Title 

Mail Code 

Phone 

Email Address 

*■» 

0 

V 

•m 

U 

: a 

: 

Mission (if applicable) 

Location 

Date 

Time 


Specify your role at time of incident (e.g. test conductor, subject, etc).. 


Specify the role of other persons involved in the incident. 


For additional information contact Dr. Frances Mount at the following address: 

Space Operations Incident Reporting, Mail Code: SP34, NASA Johnson Space Center, Houston, Texas 77058 


A-l 

















II. Description of Incident 

Use the list below to help structure y< 
Check any items under each categor; 
items, determine whether each iter 
Equipment/Systems category may be 


quipment/ Systems category may be a computer, a switch, a display, crew personal equipment, 
ategorizine, write a description of the incident below. Include information mat you feel is impo 
le incident. 


nent, a robotic arm, etc. After 
important for identification of 


1. Environment 

2. Human 


3. Equipment/Systems 

□ lighting 

□ visual 

G hygiene 

□ design 

□ noise level 

□ auditory 

□ memory 

□ labels 

□ vibration 

□ tactile 

□ mental processing 

O controls /input devices 

□ temperature 

□ physical size 

□ sensory overload 

G displays 

□ gravity 

□ mobility 

□ cultural /language 

G warnings 

□ available space 

□ endurance 

differences 

Q accuracy 

D air quality \pressure 

D strength 

□ procedures /training 

G processing speed 

□ housekeeping 

□ comfort 

n other 

D reliability 

□ other 

□ physical stress /fatigue 

n other 




III. Causes & Possible Solutions 

Using the boxes below, please check the category of the incident. Also provide information that might help identify the 
cause of the incident, ana suggestions for preventing recurrence of this incident in the future. Examples are: 

1. "Information on the display changed too rapidly to make use of it. Solution: Provide history list or data ouptuts so that I 
can review them later.” 

2. ,r Radio frequency button is often pressed when trying to adjust volume. Solution: Separate controls or provide stiffer 
resistance for button to keep inadvertent presses from activating it. 


□ Category I Minor modifications to design or training procedure needed to rectify situation. 

□ Category II Significant modifications to design or training are required. 

□ Category III Complete hardware and/or software redesign is required. 


For additional information contact Dr. Frances Mount at the following address: 

Space Operations Incident Reporting, Mail Code: SP34, NASA Johnson Space Center, Houston, Texas 77058 
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